1. Field of the Disclosure
The present disclosure relates to a scope navigation system and an image pickup calibration system for a scope navigation system. More particularly, the present disclosure relates to an apparatus, method and computer code product for accurately navigating a scope and calibrating an image pickup device for a scope navigation system.
2. Background of the Disclosure
Minimally-invasive medical procedures have become commonplace with imaging systems that allow users, such as for example, clinicians, researchers and others to see inside bodies of living organisms with amazing clarity. Users are turning to endoscopes, which is one type of scope, as a first line tool in researching, evaluating, diagnosing and/or performing medical procedures where minimal invasive intrusion is desired and/or necessary.
Endoscopes have become prevalent throughout the scientific community, with the greatest proliferation occurring in setting such as, for example, clinics, operating rooms, doctor offices, and the like. Part of the reason for the ubiquitousness of endoscopes has been because they facilitate minimally-invasive diagnostic and surgical medical procedures. Endoscopes are typically used to evaluate interior surfaces of, for example, the human body. Typically, endoscopes may be retrofitted with diagnostic and/or surgical instruments that may be remotely operated by a user.
Contemporary endoscopes may be categorized into at least two groups, including, for example, rigid endoscopes and flexible endoscopes. A typical endoscope, whether it is a rigid or flexible endoscope, includes a light delivery system to illuminate an object under inspection, such as, for example, an organ; a lens, or lens system, for projecting an image of the object; an image pickup device, such as, for example, a charge coupled device (CCOD), for recording the projected image of the object under inspection; a processor that receives image signals from the image pickup device and converts the image signals to humanly perceivable display images; and a display that displays the perceivable images.
The contemporary rigid endoscope differs from its flexible counterpart in that it uses a rigid insertion portion and transmits an image through a lens system. Typically, an optical sensor is attached to a handle in a rigid endoscope. Whereas the flexible endoscope uses a flexible insertion portion and typically transmits an image through a flexible system, such as, for example, a fiberscope. Generally, an optical sensor is placed into a distal end portion of a flexible endoscope.
As the popularity of endoscopes has grown and become commonplace in medical procedures, the complexity of the hardware and software equipment that accompanies the endoscopes has also grown. Particularly, as the quality and clarity of the picked-up images has improved, users have learned to depend on the endoscopes more than ever in accurately performing medical diagnostic and/or surgical procedures. As a result, various methodologies have been developed to correct for system imperfections pertaining to, for example, video cameras and their implementations at the micro-level, as well as the interplay between the various coordinate systems.
Three commonly used camera calibration methods are the Direct Linear Transform (DLT), R. Y. Tsai, and Z. Zhang methods. Of the three, Tsai's method is the oldest and most widely used in computer vision systems because of its effective and noteworthy performance. There are numerous implementations of the Tsai calibration method using C, C++ and other high-level computer languages. The Tsai method has been used to correct for internal camera geometric and optical characteristic imperfections (intrinsic parameters) and/or errors resulting due to the three dimensional position and orientation of the camera frame relative to a certain world coordinate system (extrinsic parameters).
Scope systems that use camera calibration and take into consideration the various coordinate systems in aiding a user in navigating the scope are sometimes referred to as augmented reality (AR) systems. AR systems are typically display-based in that the final image displayed to the scope operator is accurately depicted in terms of both its positional aspect and its time aspect.
In order to facilitate hand-eye coordination by the scope operator, display-based AR technologies have been mated to three-dimensional (3D) position sensor scope tracking technology, allowing users to accurately track movements of the endoscope while viewing a displayed image of the area sensed by the scope. However, a significant limitation of scopes remains, including AR-based endoscopes. Namely, the coordinate system of the image picked up by a scope has an imprecise relationship to the coordinate system perceived by the operator, such as, for example, the hand-eye coordination between the scope as manipulated by the user and the image displayed on the external monitor to the user.
Japanese Patent Application Publication No. 2001-187067 describes an AR-based endoscope that compensates for the various coordinate systems existing in an endoscope observation system. As illustrated in FIG. 1, Publication No. 2001-187067 describes an endoscope observation position detecting and indicating system that allows a user to accurately navigate an endoscope in real time while viewing a corresponding image that is picked up by the endoscope. The known navigation system detects the observation position of an endoscope 1. The system employs a calculation device 7 to compute the endoscope observation position by processing signals received from an optical detector 8, via signal line 9, and a sensor arm 5, via signal line 6, to detect an observation position of the endoscope 1. A monitor 10 displays an image picked up by a video camera 4. The video camera 4 and sensor arm 5 are supported by support arm 2 and guide 3. Thus, by using, for example, detector 8, and an optical sensor attached to video camera 4, the known system is able to track the position of the camera. However, the optical navigation system shown in FIG. 1, which employs an optical sensor attached to the handle, does not work for a flexible endoscope, whose distal end portion is curved.
In flexible scopes it has been a practice to insert a magnetic sensor, for example, through a scope forceps channel, or to build the sensor into a distal end portion of a scope as shown, for example, in FIG. 2. However, the offset between an optical axis of the scope and the central axis of the sensor creates a misalignment between the actual image picked up by the scope and the location of the image as sensed by the sensor.